Skip to content

GH-51029: [C++] Fix MapArray validation with unknown null count - #51093

Merged
pitrou merged 3 commits into
apache:mainfrom
AnuragRaut08:fix/maparray-cast-validity
Sep 2, 2026
Merged

GH-51029: [C++] Fix MapArray validation with unknown null count#51093
pitrou merged 3 commits into
apache:mainfrom
AnuragRaut08:fix/maparray-cast-validity

Conversation

@AnuragRaut08

@AnuragRaut08 AnuragRaut08 commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Rationale for this change

MapArray validation uses MayHaveNulls() to check that the map child and keys contain no nulls. When an all-valid validity bitmap is present with an unknown null count, MayHaveNulls() can report that the array may contain nulls even though all values are valid.

During a cast of a MapArray, this can cause validation to return an invalid status which is passed to ARROW_CHECK_OK, resulting in a process abort instead of a recoverable error.

What changes are included in this PR?

  • Use GetNullCount() instead of MayHaveNulls() when validating the MapArray child and keys.
  • Add a regression test covering an all-valid validity bitmap with kUnknownNullCount.

Are these changes tested?

Yes, with existing and additional unit tests.

Are there any user-facing changes?

Only a bugfix.

This PR contains a "Critical Fix".

This fixes a bug that can cause a process crash when casting a valid MapArray whose keys carry an all-valid validity bitmap with an unknown null count. This is easily reproduced in Python:

import pyarrow as pa
m = pa.array([{"a": "1"}, {"b": "2"}], type=pa.map_(pa.string(), pa.string()))
m.filter(pa.array([False, True])).cast(pa.map_(pa.large_string(), pa.large_string()))

@AnuragRaut08
AnuragRaut08 requested a review from pitrou as a code owner August 31, 2026 06:25
@github-actions

Copy link
Copy Markdown

⚠️ GitHub issue #51029 has been automatically assigned in GitHub to PR creator.

@pitrou pitrou left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing this @AnuragRaut08 ! See comments below.

Also, there are other uses of MayHaveNulls in array_nested.cc, shouldn't we fix them as well?

Comment thread cpp/src/arrow/array/array_nested.cc
Comment thread .gitignore Outdated
Comment thread cpp/src/arrow/compute/kernels/scalar_cast_test.cc Outdated
@github-actions github-actions Bot added awaiting committer review Awaiting committer review and removed awaiting review Awaiting review labels Aug 31, 2026

@Reranko05 Reranko05 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also fix the lint by using:-
python -m pre_commit run clang-format --files <file_path>

@AnuragRaut08
AnuragRaut08 force-pushed the fix/maparray-cast-validity branch from 0e4c2a2 to 4574e70 Compare September 1, 2026 15:49
@AnuragRaut08

Copy link
Copy Markdown
Contributor Author

Done, ran clang-format on the affected C++ files with pre-commit.

@AnuragRaut08

Copy link
Copy Markdown
Contributor Author

@pitrou I checked the other MayHaveNulls() uses in array_nested.cc. They are used for validating nullable list/map offsets when constructing arrays, rather than validating MapArray keys, so I left them unchanged.

@pitrou

pitrou commented Sep 1, 2026

Copy link
Copy Markdown
Member

@pitrou I checked the other MayHaveNulls() uses in array_nested.cc. They are used for validating nullable list/map offsets when constructing arrays, rather than validating MapArray keys, so I left them unchanged.

But the same issue applies: MayHaveNulls does not tell whether there are really nulls. So we should strengthen those checks as well.

@AnuragRaut08
AnuragRaut08 force-pushed the fix/maparray-cast-validity branch from 4574e70 to aee42e9 Compare September 1, 2026 16:17
@AnuragRaut08

Copy link
Copy Markdown
Contributor Author

Thanks, addressed. I have updated the remaining MayHaveNulls() checks in array_nested.cc to use GetNullCount() != 0 and added regression coverage for an all-valid offsets bitmap with kUnknownNullCount. The TestMapArray.* tests pass.

@pitrou pitrou left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, LGTM.

Comment thread .gitignore Outdated
MANIFEST
compile_commands.json
build.ninja
build/

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
build/

@pitrou pitrou added Critical Fix Bugfixes for security vulnerabilities, crashes, or invalid data. backport-candidate labels Sep 2, 2026
@pitrou

pitrou commented Sep 2, 2026

Copy link
Copy Markdown
Member

@AlenkaF FYI

@pitrou
pitrou merged commit a4e3909 into apache:main Sep 2, 2026
89 of 109 checks passed
@pitrou pitrou removed the awaiting committer review Awaiting committer review label Sep 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport-candidate Component: C++ Critical Fix Bugfixes for security vulnerabilities, crashes, or invalid data.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants